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The objective of the present study is to develop a measurement tool to assess 21st 
Century learning and innovation skills of primary school students. Study data was 
collected from 632 fourth grade students in five different primary schools during 
2014-2015 academic year and data obtained from 609 fourth grade students were 
utilized in the study. The scale was developed in six stages. These were; 
establishing the scale items, consultation of experts, pretest stage, determination of 
structural validity, reliability assessment, and finalization of the scale, respectively. 
Cronbach alpha reliability coefficient for the whole scale, which consisted of three 
factors, was calculated as 0.95. 0.89. As a result of the current study, a Likert-type 
21st Century learning and innovation skills scale with 39 items was developed. 20 
items of the scale were related to creativity and innovation skills, 12 were related 
to critical thinking and problem solving skills, and 7 were related to cooperation 
and communication skills. 

Keywords: primary school, 21 st century, learning and innovation skills, scale, validity 
and reliability 

INTRODUCTION 

In today’s society, which was christened as the information society where information is 
produced and spread to the large masses rapidly, information is the most significant 
phenomenon and individuals are also expected to reach that knowledge. Along with the 
technological developments, scientific innovations, increasing globalization, changing 
labor demands, economic pressures and rapid changes in competition in today’s society, 


1 This paper is based on a PhD study titled “Applying Slowmation in Science Lesson to Improve 
Students’ Learning and Innovation Skills”. This thesis study was supported by Anadolu 
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the skills required in the students to contribute to the society and increase their readiness 
for society were redefined (Levy and Murnane, 2005; Stewart, 2010; Wilmarth, 2010). 
In information society, the individuals are expected to adjust to the complex society, 
select knowledge within the information mass, analyze this knowledge to utilize it to 
solve problems of daily life and to transform it into products. Thus, individual should 
possess high level skills such as analytical thinking, inter-personal skills and 
organization of knowledge (Velez, 2012). The needs of individuals in the 21st Century 
to work, for citizenship and to realize oneself are quite different from their possessions 
in the 20th Century. 21st Century skills are different from that of the 20th Century 
primarily based on the emergence of advanced information and communication 
technologies (Dede, 2009; Shute and Becker, 2010). 

Throughout history, critical thinking and problem solving skills were the main 
components of human development such as exploration of land and the seas, discovery 
of vaccines, agricultural developments and in many other fields. However, since skills 
such as information literacy and global awareness could be found in different societies, 
these are no longer considered as new skills. Plato expressed these concepts as four 
different levels of the mind back in mid Third Century (Rotherham and Willingham, 
2009). Conceptual framework of these skills, named as 21st Century skills, were 
considered in different dimensions by P21 (Partnership for 21st Century skills), NRC, 
Metiri Group and NCREL (North Central Regional Education Laboratory) (2003), 
American Association of Colleges and Universities (2007), and OECD (2005). National 
organizations including P21 and NRC investigated the interactions between 21st 
Century skills and education and their instruction in core disciplines to identify and 
define 21st Century skills (NRC, 2006; P21, 2009). Since it is an accepted fact that the 
developments in information and communication technologies had a profound effect on 
21st Century skills, student technology standards included in the programs based on 
ISTE NETS*S (International Society for Technology Education, 2007) data were 
reviewed and organized (Dede, 2009). Educational demands and skills of the new 
century require new ways in thinking, learning and teaching (Keengwe, Onchwari and 
Wachira, 2008; Razzouk and Oroszlan, 2011). In today’s societies, there is a need for 
individuals, who could research and structure the information, with critical thinking 
skills, could resolve the problems with own knowledge and experience, are creative, 
open to innovations, and at the same time, with effective communication and 
cooperation abilities within a group. P21 (2009) called these skills, which the 
individuals should possess, 21st Century skills and classified these as “Learning and 
Innovation Skills,” “Life and Career Skills,” and “Information Media and Technology 
Skills.” According to Prensky (2001) and Rotherdam and Willingham (2010), 21st 
Century skills are not a different set of skills. Every individual is expected to have 
creative thinking and problem solving skills that are required for active participation of 
the individual in the society and the workplace. For students’ achievements in 
educational life, they should experience these skills at a very early age (Louis, 2012). 

Thus, the renewed primary school instruction programs included these skills as well. 
Primary schools have a significant function in formation of the future educational steps 
and due to the fact that the information and skills obtained in this level affect the future 
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training experiences. According to Giiltekin (2007), primary school, which is the initial 
step of primary education, sustains the society by transferring the cultural background 
and values of the society to the future generations; on the other hand, by providing basic 
information and skills for the individuals, it increases the social and cultural level of the 
society. In other words, the objective of the primary school is to increase the quality of 
life of individuals and the society by spreading education among the masses. The skills 
of “creativity and innovation,” “critical thinking and problem solving,” and 
“communication and cooperation” under learning and innovation skills coincide with the 
skills included in primary school instruction programs. It is significant to provide 
lifelong learning and 21st Century skills for the students from the primary school on 
(Shin and Lee, 2008). Along with the technological developments and changes in the 
field of education and the innovations in the content of instructional programs and in 
learning-teaching process, it became necessary to determine the level of efficacy of the 
students in these skills. Several measurement tools were developed to measure these 
skills of students in the literature. 

Torrance (1966) developed a scale influenced by Guilford’s Cognitive Learning Ability 
test, one of the Verbal A and Verbal B forms, designed to measure creative thinking 
skill. This scale is a paper-and-pencil test that measures individual creativity. This test, 
one of the most known and frequently applied tests based on divergent thinking, was 
restructured several times until it took the form used today (Torrance, 1990). “How 
Creative Are You?” creativity scales developed by Raudsepp (1977) and Whetton and 
Cameron (2002) were translated into Turkish and used in several studies. 

“California Critical Thinking Tendency Scale” developed by Facione and Giancarlo 
(1992), a test developed by Ennis-Weir (1991) to measure the logical dimension of 
critical thinking, Ozdemir’s (2005) “Critical Thinking Skill Scale,” another “Critical 
Thinking Skill Scale” developed by Demir (2006), and Watson-Glaser’s (1980) 
“Critical Reasoning Power Scale” were used in several studies. 

“Problem Solving Inventory” developed by Heppner and Peterson (1982) and other 
problem solving inventories developed by Taylan (1990), Ge (2001), Yaman (2003), 
(”am and Tiimkaya (2008) to measure problem solving skill perceptions of high school 
students, pre-service teachers and adults and were used in several studies. Serin, Bulut- 
Serin and Saygili (2010) developed an inventory consisting of three factors and 24 
items; “confidence” (12 items), “self-control” (7 items) and “avoidance” (5 items), to 
measure the self-perception of primary school students on their problem solving skills. 

Kang et al. (2010) developed a 21st Century skills scale for high school and middle 
school students that included cognitive, affective and sociocultural sub-dimensions. 
Kang et al. (2012) later adapted this scale for the primary school level students. 
Furthermore, Deborah (2012) developed a scale for teachers for determination of 
utilization of 21st Century skills at schools, which included “education and instruction,” 
“professional community” and “school culture” sub-dimensions. 

There are limited number of original scales and Turkish adaptations that measure all of 
these skills; however there are scales that measure the skills separately. Most of the 
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scales were developed for use with adolescents and adults (("am and Ttimkaya, 2008; 
Deborah, 2012; Kang et al., 2010; Ozdemir, 2005). As a result of innovations in 
educational content and learning-teaching process, a need for measurement tools has 
arisen to determine the levels of these skills that the students have. It was observed that 
the scales developed to measure all skills in middle school, high school and college 
levels did not include psychometric features required to measure skills and behavior of 
primary school students. The lack of a measurement tool designed for primary school 
students was the starting point of the present study. To fill this gap, it was considered 
that the development of “21st Century Learning and Innovation Skills" scale would 
contribute to the field and could help determine whether the individuals possess these 
skills at an early age, and could serve as a resource for experts and educators, who 
would conduct research in this field. Thus, the present study aimed to develop “21st 
Century Learning and Innovation Skills” scale, which would determine “Creativity and 
Innovation,” “Critical Thinking and Problem Solving,” and “Cooperation and 
Communication” skills of primary school students. In this framework, the current study 
scrutinized the validity and reliability of the scale that would be developed to determine 
“21st Century Learning and Innovation Skills” of fourth grade primary school students. 

METHOD 

The present research is a scale development study. The developmental stages for the 
“21st Century Learning and Innovation Skills” scale and the characteristics of the study 
group are detailed below. 

Study Group 

The selection of the sample in scale development studies is still a matter of debate. In 
the literature, while Comrey and Lee (1992) considered a sample size of 100 
participants as insufficient, 200 participants as average, 300 participants as good, 500 
participants as very good, and 1000 or more participants as perfect. Field (2009) stated 
that at least 300 participants are required to conduct factor analysis. Thus, the universe 
of the present study included fourth grade students that attended the primary schools in 
Adiyaman province of Turkey during 2014-2015 academic year. Sample of the study 
consisted of 632 students attending the fourth grade in five primary schools located in 
Adiyaman city center, determined with random sampling from different socio-economic 
areas. The forms completed by 23 students out of 632 were excluded from the study due 
to reasons such as missing answers, selecting more than one multiple choice answers. 
The responses excluded by students were not included in the study and responses by the 
remaining 609 students were analyzed. 

Scale Development Process 

The “21st Century Learning and Innovation Skills” scale was developed in six stages of 
establishing the items of the scale, consultation of experts, pretest stage, determination 
of structural validity, reliability assessment, and finalization of the scale, respectively. 

Establishing the Items of the Scale 

The objective of the study is to determine the degree that fourth grade primary school 
students had 21st Century learning and innovation skills. A multidimensional approach 
was followed in the study and a literature review was conducted by considering all 
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phases of 21st Century learning and innovation dimension. Preliminary items were 
identified by taking into account the scale items that existed in the reviewed studies in 
the literature. 

Expert Opinion 

Draft scale consisted of 60 items. Opinions of five experts in the field were consulted on 
the draft scale. The experts scrutinized the items based on their suitability for the 
dimension, comprehensibility of the expressions, whether the items concerning the skills 
under each dimension reflected all the characteristics of the related skill, and whether 
these were suitable for fourth grade students. As a result of expert assessments, four 
statements that were not suitable for the level of students and comprehensibility of the 
statements were excluded from the scale, while five statements that did not reflect the 
related skills completely were edited. 

Pre-Test Stage 

To determine the answering duration and comprehensibility of the scale, the 56-item 
scale was applied to 30 fourth grade primary school students in the pre-test stage. As a 
result, two statements that were not understood clearly by the students were edited. 56- 
itern draft scale was thus finalized. During the following phase, the scale was applied to 
the students and scores obtained from the scale were analyzed. 

Determination of Structural Validity 

Exploratory factor analysis was conducted on obtained data to determine the structural 
validity of the scale. Exploratory factor analysis (EFA) is an analysis technique that 
enables researchers to identify and group the items that measure the same structure or 
quality among the items determined previously and explains the measurement with these 
small number of superstructures (factors) (Buytikozturk, 2007). Exploratory factor 
analysis was conducted initially, followed by the confirmatory factor analysis to 
establish the structural validity of the scale. Confirmatory factor analysis (CFA) is a 
statistical method used to determine whether the variable groups within predetermined 
factors would sufficiently be represented by these factors (Buyiikozturk, 2007; 
Tabachnick and Fidell, 2007). In this process, primarily Kaiser-Meyer-Olkin (KMO) 
and Bartlett Sphericity tests results were examined, followed by the common factor 
variance values for the items, eigenvalue line graph, principal components analysis 
results, and “verimax” rotation technique results, conducted to obtain interpretable 
variables. As a result of the interpretation of the above-mentioned data, the scale was 
finalized. 

Assessment of Reliability 

One of the main problems that scales used in educational research face is the issue of 
reliability. In this stage, item-total test score correlations between the items and 
Cronbach alpha reliability coefficient value was calculated and analyzed to test the 
reliability of the scale. Cronbach alpha reliability coefficient value is a measure of 
internal consistency between the scale test scores and values of 0.70 or higher are 


International Journal of Instruction, January 2016 • Vol.9, No.l 





138 


A Scale Development for 21st Century Skills of Primary ... 


accepted as sufficient for test reliability. Item-total test score correlation is used to 
explain the relationship between the item score and test items total score. A high and 
positive item-total test score correlation demonstrates the internal consistency of the 
scale (Biiyukozturk, 2007). 

Finalization of the Scale 

Certain items were omitted from the scale as a result of the processes explained in the 
above stages and the scale was finalized. 

FINDINGS 

Findings are grouped under three titles for a better systematic presentation: 

1. Assessment of data for factor analysis suitability 

2. Analysis of the structural validity of the scale 

3. Analysis of the reliability of the scale 

Assessment of data for factor analysis suitability 

Factor analysis is a multivariate statistical method that aims to discover smaller number 
of conceptually significant new variables (factors, dimensions) by aggregating greater 
number of variables (Biiyukozturk, 2007). Initially, the suitability of the date for EFA 
was controlled. For this purpose, missing data in the measurement tools, reverse items, 
normality of the dataset, relationships between the variables, sample size, and sample 
suitability were controlled using KMO and Bartlett sphericity tests (Biiyukozturk, 2007; 
Field, 2009; Tabachnick and Fidell, 2007). Items 19, 23, 26, 28, 41, 42, 45, 53, which 
contained negative statements were reversed. Results demonstrated that Bartlett test was 
significant (p<0.05) and KMO value was greater than 0.50 (KMO= .943). Related 
literature would show that a KMO value of 0.60 is considered as moderate; 0.70 as 
good; 0.80 as very good; and 0.90 as perfect (Sharma 1996, Bryman & Cramer, 1999; 
§eker, Deniz and Gorgen, 2004). Table 1 demonstrates Bartlett test and KMO results. 

Table 1; Kaiser-Mayer-Olkin (KMO) sample measurements and Bartlett’s test results 


Kaiser-Mayer-Olkin (KMO) 

0,943 



Bartlett’s Test Approximate Chi-Square 
Value 

11459,213 

Sd=1540 

p= 0.00* 


Data presented in Table 1 demonstrates that the obtained data were suitable for 
exploratory factor analysis as the KMO coefficient of 0.943 reflected that sample size 
was perfect and Bartlett’s test results reflected a correlation between the items of the 
scale. 

Analysis of the structural validity of the scale 

EFA was conducted with the data obtained from the first 304 students, and CFA was 
conducted using the dataset for the second 305 students in the study. According to 
Worthington and Whittaker (2006), EFA and DFA should be conducted with different 
samples. Thus, 609 students were randomly divided into two groups and EFA analysis 
was conducted on one group, whereas CFA analysis was conducted on the other. 
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Findings for the Exploratory Factor Analysis (EFA) 

In EFA, the qualification of the sample was analyzed using Kaiser-Meyer-Olkin (KMO) 
value and the suitability of the data for factor analysis was analyzed using Bartlett’s 
Sphericity value. It was determined that KMO coefficient for the data was .943 and 
Bartlett’s Sphericity test jl value was 11,459.213 and both values were significant (p < 
.05). Factor analysis for the scale was conducted with Principal Components Analysis 
technique. To support the results and to make an accurate decision on the number of 
factors in the scale. Scree graph, which is based on the factor eigenvalues, was also 
analyzed (Buyukozturk, 2007). As a result of the analyses and based on the 
discontinuities in the graph, a three-factor structure was determined for the scale. The 
break point observed in the Scree plot, which is related to the number of factors, also 
indicated a three-factor structure. As a result of the initial EFA analysis, 56 items were 
grouped under eight factors to explain 62.708% of the total variance. The minimum 
value in determination of the factor loads of the items was set as .30 (Pallant, 2001). At 
this point, the items that did not have a difference of at least .10 between the factor load 
values that the item received in more than factors (i.e. overlapping items) were excluded 
from the scale starting from the least necessary items. Pallant (2001) also indicated in 
the common variance (communalities) table that the values below .30 were not 
compatible with other items in the same factor. Thus, common variance table was 
controlled when each item was excluded; however a value below .30 was not 
encountered in any phase. The scale was finally formed with three factors including 39 
items. In the next step, to group the items that demonstrated high level of relationship 
with each other, it was decided to implement Varimax Orthogonal Rotation technique 
(Buyukozturk, 2007; Kalayci, 2005). Item analysis of the scale supported the findings 
that the scale was three-dimensional as a result of the Varimaz Orthogonal Rotation as 
well. It was identified that the total variance that the scale explained with the items 
grouped under three factors was 61.302%. Concurrently, Cronbach reliability coefficient 
for the scale was determined as a = .955. Rotated factor loads for the scale, the 
variances explained by different factors and reliability information are presented in 
Table 2. 


Table 2: Factor load values and common factor variance for the 21st Century learning 
and innovation skills scale 



Items 

Common Factor 

Rotated Factor Load Value 



Variance 

Factor 1 Factor 2 Factor 3 

g 

15 

0,522 

0,561 

.o 

17 

0,747 

0,614 

> 

Q 

18 

0,679 

0,582 

G 

G 

19 

0,539 

0,535 

-a 

Ill 

0,540 

0,658 

G 

112 

0,620 

0,727 

■w' 

113 

0,380 

0,606 


114 

0,513 

0,681 

a 

u 

118 

0,707 

0,807 

120 

0,667 

0,791 
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136 

0,530 

0,692 




138 

0,564 

0,742 




139 

0,638 

0,771 




140 

0,687 

0,807 




143 

0,609 

0,754 




144 

0,658 

0,794 




145 

0,536 

0,681 




146 

0,513 

0,691 




147 

0,671 

0,793 




13 

0,452 


0,501 


g 

14 

0,635 


0,612 


<D 

121 

0,762 


0,871 


a 

123 

0,431 


0,653 


Oh 

W) bj) 

125 

0,712 


0,834 


g c 
'm ’> 

127 

0,630 


0,783 


c 1—1 
•3 o 

129 

0,599 


0,754 


cn 

H 

130 

0.806 


0,892 


13 

o 

131 

0,456 


0,666 


•n 

132 

0,690 


0,828 


u 

133 

0,750 


0,860 



134 

0,755 


0,861 



148 

0,669 



0,773 


149 

0,704 



0,760 

'g c 

G O 

150 

0,692 



0,801 

& . y 

G fl 

151 

0,653 



0,774 

o .y 
g 

152 

0,619 



0,745 

§ i 

153 

0,381 



0,600 

° 1 

154 

0,650 



0,786 

6 u 







Total variance explained %61,302 

%27,730 

%21,819 

% 11,754 


Cronbach alpha a=.955 

a=.958 

a=,943 

a=.896 


For an item to be placed under a factor, it should have a factor load of at least .30 
(Pallant, 2001). As could be observed in Table 2, the first factor included 20 items with 
factor loads that varied between .53 and .80; the second factor included 12 items with 
factor loads that varied between .50 and .89; and the third factor included 7 items with 
factor loads that varied between .60 and .80. It was observed that all factors explained 
61.30% of the total variance. The first factor explained 27.73% of the total variance and 
named as “creativity and innovation” with the guidance of the previous studies in the 
literature. The second factor explained 21.81% of the total variance and named as 
“critical thinking and problem solving.” The third factor explained 11.75% of the total 
variance and named as “cooperation and communication.” Analysis results demonstrated 
that all items were under the planned dimension with the exception of three items. Two 
items (138 and 144) were considered under critical thinking and problem solving 
dimension during the planning phase, however EFA results determined that these were 
under creativity and innovation dimension. One item (19) was considered under 
cooperation and communication dimension during the planning phase, however EFA 
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results determined that this item was under critical thinking and problem solving 
dimension. Post-analysis expert opinion agreed that these 3 items could be placed under 
the dimensions determined by EFA results. 

Findings for Confirmatory Factor Analysis (CFA) 

CFA was conducted to test whether the data collected from the second study group 
would confirm the scale structure with 39 items and three factors obtained as a result of 
EFA. Goodness of fit indices obtained from the confirmatory factor analysis conducted 
on the three-factor structure are given below before application of any modifications on 
the model: [%2/sd=1.66 (p=.000); GFI=.84; AGFI=.82; CFI=.98; NFI=.95; NNFI=.98; 
IFI=.98; RMSEA=.047; SRMR=.051; PNFI=.89 ve PGFI=.75]. Connections between 
13 and 121, and 14 and 123 items, proposed by the modification indices that were 
obtained as a result of the analysis were analyzed. As the theoretical analysis concluded 
that these items measured similar conditions and there could be a hidden relationship 
among them, modification suggestion was considered. Fit indices for the modes 
obtained as a result of the modification are displayed in Table 5. Perfect and acceptable 
fit criteria observed in Table 5 demonstrated that the goodness of fit level of the three- 
factor model obtained as a result of CFA was sufficient. The findings of the study 
reflected the following value: %2/df=1.42. When the value of this ratio m is two or 
below, it means that the model is a good model; and when the ratio is five or lower, it 
reflects a model with an acceptable goodness of fit (§im§ek, 2007). Thus, the value 
obtained indicated that the model was good. In addition, the value ranges of GFI 
(Goodness of Fit Index), AGFI (Adjusted Goodness of Fit Index), CFI (Comparative Fit 
Index), Normed Fit Index (NFI, Non-Normed Fit Index (NNFI), Incremental Fit Index 
(IFI) RMSEA (Root Mean Square Error of Approximation), and SRMR (Standardized 
Root Mean Square Residual), Parsimony Normed Fit Index (PNFI) and Parsimony 
Goodness of Fit Index (PGFI) fit indices were referenced based on the resources by 
Brown (2009), Hu & Bentler (1999), Joreskog and Sorbom (2000), Kline (2011), 
Raykov and Marcoulides (2006), Tabachnick and Fidell (2007), Thompson (2004), 
and are presented in Table 3 along with the final fir index values for the model. 


Table 3: Fit statistics value ranges and values for the model analyzed in the study 


Analyzed 

Fit Indices 

Perfect 

Fit Criteria 

Acceptable 

Fit Criteria 

Obtained 

Fit Indices 

Result 

%2/sd 

0 < %2/sd < 2 

2 < %2/sd <3 

1.42 

Perfect fit 

GFI 

.90 < GFI < 1.00 

,85< GFI < 90 

.86 

Acceptable fit 

AGFI 

.90 < AGFI < 1.00 

,80< AGFI < .90 

.84 

Acceptable fit 

CFI 

.95 < CFI < 1.00 

.90 < CFI < .95 

.99 

Perfect fit 

NFI 

.95 < NFI < 1.00 

.90 < NFI < .95 

.96 

Perfect fit 

NNFI 

.95 < NNFI < 1.00 

.90 < NNFI < .95 

.99 

Perfect fit 

IFI 

.95 < IFI < 1.00 

.90 < IFI < .95 

.99 

Perfect fit 

RMSEA 

.00 < RMSEA < .05 

.05 < RMSEA < .08 

.037 

Perfect fit 

SRMR 

.00 < SRMR < .05 

.05 < SRMR <.10 

.053 

Acceptable fit 

PNFI 

.95 < PNFI < 1.00 

.50 < PNFI < .95 

.89 

Acceptable fit 

PGFI 

.95 < PGFI < 1.00 

.50 < PGFI < .95 

.73 

Acceptable fit 


%2=992.24 sd=691 
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As could be observed in Table 3, the model is at an acceptable level. Standardized 
factor loads for the final model varied between .27 and .74. An analysis of the findings 
showed that t-test values varied between 5.47 and 13.93 for creativity and innovation 
sub-scale; between 4.76 and 13.43 for critical thinking and problem solving sub-scale; 
and between 3.98 and 13.93 for cooperation and communication sub-scale. Calculated t 
values of 1.96 and higher reflects a .05 level significance, 2.58 and higher reflects a .01 
level significance (Joreskog and Sorbom, 2000; Kline, 2011). Thus, it was determined 
that all t values obtained as a result of CFA were significant at .01 significance level. 
Insignificant t values indicate that these items should be excluded from the model or it is 
considered as the number of participants was insufficient for the factor analysis (Byrne, 
2010). Therefore, the obtained t values as a result of CFA confirmed that the number of 
participants in the study was sufficient for factor analysis and there were no items that 
should be excluded from the model. 

Analysis of the reliability of the scale. 

The scale was finalized with 39 items as a result of the conducted analyses. Initially, 
item-total correlations were calculated to determine whether each item in the scale 
measured the factor it aimed to measure, and the efficiency of the items in differentiating 
individuals based on the factor they measure. Cronbach alpha internal consistency 
coefficient was calculated to determine the reliability of the scale. Conducted reliability 
analyses produced a Cronbach alpha reliability coefficient of 0.955 for the whole scale. 
Buyiikozturk (2007) indicated that a scale with a reliability coefficient of over 0.70 
could be considered reliable. However, according to §encan (2005), a coefficient of 
0.70 or over is sufficient for scientific studies, but the scales for use in ability research 
should have a reliability coefficient of at least 0.85. Evaluation of factor-based 
Cronbach alpha coefficients would demonstrate that Cronbach alpha coefficient was 
0.958 for Factor 1; 0.943 for Factor 2; and 0.896 for Factor 3. The item-total 
correlations for the scale varied between .381 and .954, thus it could be stated that the 
scale had a consistent structure based on the items as well. Item analysis results shows 
that item-total test correlations for creativity and innovation factor varied between r = 
.49 and r = .72. item-total test correlations for critical thinking and problem solving 
factor varied between r = .38 and r = .72. For cooperation and communication factor, 
item-total test correlations varied between r = .45 and r = .95. Item-total correlations of 
.30 and higher could be used as an evidence for the validity of scale items (Nunnally and 
Bernstein, 1994). Item-total test correlations for the study were over r = .30 for each 
item. Thus, it could be concluded that the scale was set to measure the factors it aimed 
to measure. 

CONCLUSION AND SUGGESTIONS 

The significance of 21st Century learning skills in education assigns a particular 
importance to the development of scales to measure these skills and accurate 
measurement of these skills. Skills that are considered within "learning and innovation” 
dimension of 21st Century skills, which ought to be developed during primary 
education, were scrutinized in the present study and a scale that included the skills in 
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this dimension was developed. As a result of the literature review of studies conducted 
on 21st Century skills, it was determined that there was not a full consensus on the skills 
that learning individuals should achieve, and there were differences in certain 
dimensions. Sub-dimensions of 21st Century learning and innovation skills could be 
classified as creativity and innovation, critical thinking and problem solving, cognitive 
skills or skills related to individual’s self, and cooperation and communication or inter¬ 
personal skills (Kyllonen, 2012). Kang et al. (2010; 2012) developed a primary and high 
school level scale in cognitive, affective, and socio-cultural dimensions. It was observed 
that most of the studies on 21st century skills were international studies, while most 
national studies were adaptations. In the present study, a new scale was developed based 
on P21 (2009) learning and innovation dimension and the skills accepted in ISTE 
NETS*S (2007). As a result of the factor analysis conducted to determine the structural 
validity of the scale developed in the present study, it was concluded that the final 39- 
item scale could measure a specific structure. Results of the factor analysis demonstrated 
that the scale had a structure composed of three sub-dimensions. These factors were 
named as “creativity and innovation,” “critical thinking and problem solving,” and 
“cooperation and communication” based on the nomenclature used in the literature 
(P21, 2009; ISTE NETS*S, 2007). This scale is significant in the sense that all previous 
studies in the literature considered these skills separately, and the present scale 
combined those under three factors as 21st Century learning and innovation skills scale. 
Similarly, the scale developed by Kang et al. (2010; 2012) classified these skills under 
cognitive, affective and sociocultural sub-dimensions. Validity and reliability findings 
for the developed scale demonstrated that the scale had an acceptable level of reliability 
both as a whole and for individual factors. It was found that scale items both measured 
the characteristics it was aimed to measure and could differentiate between the 
individuals that had the measured characteristic and those who did not. Content validity 
of the scale was conducted with expert opinion, while structural validity tests were 
conducted using exploratory and confirmatory factor analyses. It was determined that 
standardized factor loads of the scale were at a sufficient level and t values were 
significant. Goodness of fit indices used in the model analysis demonstrated that the 
fitness between the data and the model structure was acceptable. 

The developed scale could be used to determine whether primary school students 
possessed 21st Century skills. The scale was applied at fourth grade level and validity 
and reliability studies were also conducted at the fourth grade level. The validity and 
reliability of the scale could be tested in different grade levels. 
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Turkish Abstract 

21. Yiizyil Ogrenme ve Yenilenme Becerisi OReginin Geli§tirilmesi: Gegerlik ve Giivenirlik 
Gah;masi 

Bu 9 ali§manm amaci, ilkokul ogrencilerinin 21. yiizyil ogrenme ve yenilenme becerilerini 
belirlemeye yonelik oleine araci geli§tirmektir. Arasjtirma verileri 2014-2015 Egitim-Ogretim Yill 
Giiz Donemi’nde be§ farkli ilkokuldan toplam 632 ogrenciden toplanmi§, 609 ogrencinin verileri 
dikkate alinmi§tir. 01?ek geli§tinne alti a§amadan olu§mu§tur. Bunlar sirayla 61?ek maddelerinin 
olufjturulmasi, uzman g6rii§iine ba§vurulmasi, on deneme a§amasi, yapi gecerliligi tespit a§amasi, 
giivenirlik hesaplama a§amasi ve oleege son §eklinin verilmesi olmak iizere alti a§amali bir siire? 
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izlenmiiytir. U 9 faktorden olu§an 61?egin tamaminin Cronbach alfa guvenirlilik katsayisi 0.95 
olarak hesaplanmi§tir. Her bir faktor igin Cronbach alfa guvenirlik katsayisi 0.95, 0.94 ve 0.89 
olarak hesaplanmi§tir. Yapilan 9 ali§ma sonucunda, U 9 IU likert tipi 39 maddelik 21. yiizyil 
ogrenme ve yenilenme becerisi ol 9 egi geli§tirilmi§tir. Ol 9 ek maddelerinin 20 ’si yaraticilik ve 
yenilenme becerisi ile ilgili, 12 ‘si ele§tirel du§iine ve problem 9 ozme becerisi ile ilgili, 7’si 
i§birligi be ileti§im becerisi ile ilgilidir. 

Anahtar Kelimeler: ilkokul, 21. Yiizyil, ogrenme ve yenilenme becerileri, ol 9 ek, ge 9 erlik ve 
giivenirlik 


French Abstract 

Un Developpement d'Echelle pour les Competences du 21e siecle d'Etudiants d'Ecole 
primaire: une Validite et une Etude de Fiabilite 

L'objectif de l'etude presente est de developper un outil de mesure pour evaluer l'apprentissage du 
21e siecle et les competences d'innovation d'etudiants d'ecole primaire. Les donnees d'etude ont 
ete rassemblees de 632 etudiants de CMl(de quatrieme annee) dans cinq ecoles primaires 
differentes pendant 2014 - l'annee d'universitaire de 2015 et des donnees obtenues de 609 
etudiants de CMl(de quatrieme annee) ont ete utilises dans l'etude. L'echelle a ete developpee 
dans six etapes(scenes). Ceux-ci etaient; en etablissant les articles d'echelle, consultation d'experts, 
pretest etape(scene), determination de validite structurelle, evaluation de fiabilite et finalisation 
de l'echelle, respectivement. Cronbach le coefficient de fiabilite alpha pour l'echelle entiere, qui a 
consiste en trois facteurs, a ete calcule comme 0.95. 0.89. En consequence de l'etude actuelle, un 
apprentissage du 21e siecle Likert-type et une echelle de competences d'innovation avec 39 
articles ont ete developpes. 20 articles de l'echelle ont ete relates a la creativite et les competences 
d'innovation, 12 ont ete relatees a la pensee critique et les competences de resolution de 
problemes et 7 ont ete relatees a la cooperation et des competences de communication. 

Mots Cles: ecole primaire, le 21e siecle, apprenant et competences d'innovation, echelle, validite 
et fiabilite 


Arabic Abstract 
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